Does decision documentation help junior designers rationalize their decisions? A comparative multiple-case study

Software architecture design is challenging, especially for junior software designers. Lacking practice and experience, junior designers need process support in order to make rational architecture decisions. In this paper, we present the results of a comparative multiple-case study conducted to find out if decision viewpoints from [1, 2] can provide such a support. The case study was conducted with four teams of software engineering students working in industrial software projects. Two of the four teams were instructed to document their decisions using decision viewpoints; the other two teams were not instructed to do so. We observed the students for a period of seven weeks by conducting weekly focus groups and by analyzing their work artifacts and minutes. Our findings suggest that junior designers who use decision viewpoints are more systematic in exploring and evaluating solution options. However, the decision viewpoints did not help them in managing requirements and complexity. 1. Motivation and Background Software architecture design is a demanding task which requires designers to find optimal solutions within a specified timeframe for often vaguely defined requirements, while managing risks, regarding constraints, and taking business drivers into account. There is a steep learning curve to becoming a good architect: junior software designers usually have to endure extensive periods of learning, going through numerous painful trial and error attempts when making architecture decisions. Before becoming software architects, junior software designers need to develop a) a certain body of knowledge, and b) the cognitive skills for systematically reasoning about architecture decisions. These two factors are important for making rational decisions. A rational decision is a decision based on the application of reason. A rational decision deliberates the benefits and drawbacks of the available design options, while taking requirements and other project constraints into account. Junior software designers need guidance to handle the inherent complexity of rational decision making, especially with software architecture issues. Explicitly modeling ∗Corresponding author Email addresses: [email protected] (U. van Heesch ), [email protected] (P. Avgeriou), [email protected] (A. Tang) Preprint submitted to Elsevier January 20, 2013 architecture decisions during the design process may provide such a guidance. Although being widely overlooked in software engineering education, the treatment of architecture decisions as first class entities has gained increasing attention in the software architecture research field and also in industrial practice. In recent years, many authors have stressed the importance of thoroughly documenting architecture decisions in software projects (e.g. [3, 4, 5, 6]). Initially, the main perceived benefit of documenting architecture decisions was to share a common understanding of the developed architecture between stakeholders like architects, developers, and customers [3, 7], primarily to ease change, maintenance, and evolution of the architectural design. Kruchten later stressed that the modeling of (potential) decisions, particularly their dependencies and interrelations, can also support the architect when reasoning about the decisions [4]. In our previous work, we developed a documentation framework for architecture decisions that addresses many stakeholder concerns in architecture decisions [1, 2]. Using the conventions of the international architecture description standard ISO/IEC/IEEE 42010 [8], the framework provides five viewpoints for architecture decisions, each of which being designed to address different decision-related concerns: Decision Forces Viewpoint: It makes explicit the relationships between architectural decisions and the forces that influenced the architect when making the decisions out of multiple alternatives. In this context, a force is “any aspect of an architectural problem arising in the system or its environment ( operational, development, business, organizational, political, economic, legal, regulatory, ecological, social, etc.), to be considered when choosing among the available decision alternatives.” [2]. Decision Relationship Viewpoint: It makes explicit the relationships between architecture decisions (e.g. depends on, caused by, or is alternative to). Decision Chronology Viewpoint: It shows the evolution of architecture decisions over time. Decision Stakeholder Involvement Viewpoint: It describes the roles of specific stakeholders in the decision-making process, capturing which stakeholders proposed, confirmed, or validated specific decisions. Decision Detail Viewpoint: It gives detailed information about single architecture decisions, including a comprehensive description of the chosen architectural solution and the rationale for choosing this solution. Building on the idea that modeling decisions supports the design process of the architecture, we conjecture that, besides being a useful tool for professional architects, decision viewpoints can particularly guide junior software designers, helping them to make rational decisions. The question is in which areas of software architecting can decision viewpoints help to guide designers. In this paper, we report on a comparative multiple-case study conducted with four groups of senior software engineering students (near graduation), to find out if modeling design decisions supports them in following a rational design process. We selected three decision viewpoints from our framework that particularly frame concerns related to decision making support: the decision detail viewpoint, the decision relationship viewpoint 2 (both from [1]), and the decision forces viewpoint (defined in [2]). The results show that particularly the decision forces viewpoint and the decision relationship viewpoint supported the students to systematically identify and evaluate multiple decision alternatives for the design problems. The rest of this paper is organized as follows. Section 2 presents the design of the study including the study goal, research conjecture and response variables. Section 3 reports on the data analysis and interpretation. In Section 4, we discuss potential threats to the validity of our findings. We discuss related work in Section 5 and present our conclusions in Section 6.